Local Affine Approximators for Improving Knowledge Transfer
نویسنده
چکیده
The Jacobian of a neural network, or the derivative of the output with respect to the input, is a versatile object with many applications. In this paper we discuss methods to use this object efficiently for knowledge transfer. We first show that matching Jacobians is a special form of distillation, where noise is added to the input. We then show experimentally that we can perform better distillation under low-data settings. We also show that Jacobian-penalty regularizers can be used to improve robustness of models to random noise.
منابع مشابه
A Unique Investigation of Knowledge Transfer Practices within the Muslim Business Community in Edinburgh
The aim of this research was to understand the dynamisms relating to knowledge transfer processes and practices, which exist within a business community in Edinburgh Scotland. The methodology used within this study was a uniquedesign methodology to include 20 in-depth interviews with participants from four local organizations. The sample group consisted of males aged between 20 and 70. The samp...
متن کاملImproving the natural convective heat transfer of a rectangular heatsink using superhydrophobic walls: A numerical approach
The effect of utilizing superhydrophobic walls on improving the convective heat transfer in a rectangular heatsink has been studied numerically in this paper. The vertical walls were kept at isothermal hot-and-cold temperatures and horizontal walls were insulated. The boundary condition on the walls was: no-slip for regular, and slip (with slip length of 500 µm) for superhydrophobic walls. By c...
متن کاملGolf Coaches Use Tacit Knowledge to Improve Transfer Performance: Understanding the Mediating Role of Transfer Capacity in Golf Coaching
Background. Golf is relatively new in China, and overall playing skills are low. Although golf coaches are vital in improving golf skills, their teaching skills are underdeveloped. Objectives. This study aims to explore how the coach’s tacit knowledge affects the transfer performance by exploring the factors that affect the teaching ability of Chinese golf coaches. It aims to improve the teach...
متن کاملRule Extraction for Transfer Learning
Typically rule extraction is done for the purposes of human interpretation. However, there are other possible applications of rule extraction. One practical application is transfer learning, in which knowledge learned in one task is used to aid in learning a related task. The extracted rules, which explain the learned solution to the first task, can be considered advice on how to approach the s...
متن کاملAN OBSERVER-BASED INTELLIGENT DECENTRALIZED VARIABLE STRUCTURE CONTROLLER FOR NONLINEAR NON-CANONICAL NON-AFFINE LARGE SCALE SYSTEMS
In this paper, an observer based fuzzy adaptive controller (FAC) is designed fora class of large scale systems with non-canonical non-affine nonlinear subsystems. It isassumed that functions of the subsystems and the interactions among subsystems areunknown. By constructing a new class of state observer for each follower, the proposedconsensus control method solves the problem of unmeasured sta...
متن کامل